Pragmatics of Virtual Machines for High-Performance Computing: A Quantitative Study of Basic Overheads

Heterogeneous administrative domains, operating systems (OS), and libraries make it difficult for computational scientists to fully utilize metacomputers and grids. Dealing with the presence or absence of features on, say, different clusters, adds complexity. How can a user or high-performance computing (HPC) application abstract out the heterogeneity? One possible solution is to use a virtual machine (VM) environment that supports guest operating systems and virtual disks. But, over the decades, VMs have sometimes suffered from performance overheads and limited platforms on which they can run. Through a simple quantitative study, we show that recent improvements in software and hardware support reduces the overheads for HPC applications (e.g., GROMACS, BLAST, HMMer) to under 6% for compute-intensive jobs, but 9.7% or higher for more I/O-intensive jobs, on our x86-based platform. We also argue for qualitative and pragmatic benefits of using VMs for HPC, including ease of deployment, improved functionality, and the ability to run jobs on more systems than would normally be accessible. While not perfect, VMs are emerging as a pragmatic tool in HPC. Keywords—metacomputing, grid computing, virtual machine (VM), bioinformatics, GROMACS (molecular dynamics simulation), benchmarking, file systems